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(57) Abstract 



A color image processing method for retrieving a color feature descriptor for describing color features of an image is provided. The 
color image processing method includes the steps of; a) obtaining color vectors of an input image; b) classifying the color vectors to obtain 
dominant colors of the input image and the ratios thereof; and c) representing the dominant colors and the ratios thereof as a color feature 
descriptor of the input image. The color image processing method is applied to an object-based image processing method, thereby allowing 
fast search and retrieval of multi-media contents. 
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COLOR IMAGE PROCESSING METHOD AND APPARATUS THEREOF 
Technical Field 

5 The present invention relates to a color image processing method and 

apparatus, and more particularly, to a color image processing method for retrieving 
a color feature descriptor used in indexing and searching a color image. 

Among visual features for describing multi-media contents, color is the most 
dominant feature. According to conventional color image processing methods, a 
color histogram is used for expressing the color information of an image. However, 
the conventional color image processing methods using a color histogram consisting 
15 of 1024 bins have drawbacks in that the computational complexity of image 
processing steps for describing an image is high and much processing time is 
required. 

20 Disclosure of the Invention 

To solve the above problem, it is an object of the present invention to provide 
a color image processing method which can reduce the computational complexity and 
processing time. 

It is another object of the present invention to provide a computer readable 
25 medium having a program executable by a computer to perform the color image 
processing method. 

It is still another object of the present invention to provide a color image 
processing apparatus for performing the color image processing method. 

A feature of the present invention is embodied by a color image processing 
30 method includes the steps of (a) obtaining color vectors of an input image, (b) 
classifying the color vectors to obtain dominant colors of the input image and the 



WO 00/46748 PCT/KR00/00089 

2 

ratios thereof, and (c) representing the dominant colors and the ratios thereof as a 
color feature descriptor of the input image. 

The color vectors are preferably quantized color vectors and the ratios are 
preferably percentiles. 
5 The color image processing method may further include the step of (e) 

combining the quantized color vectors and the color feature descriptor and 
representing the combination result as the whole image. 

Also, before the step (b), there may be further included the step of performing 
a predetermined filtering process for smoothing of the input image. Alternatively, 
10 before the step (b), the method may further include the step of performing a 
predetermined filtering process for noise removal of the input image. 

Also, before the step (b), the method may further include the steps of 
analyzing the probability of pixels in the filtered image being noisy pixels and 
applying appropriate weights thereto, and applying a general Lloyd algorithm to the 
15 color vectors corresponding to the weighted pixels to perform color quantization. 

According to another aspect of the present invention, there is provided a color 
image processing method for retrieving a color feature descriptor for describing color 
features of an image, the method including the steps of (a) segmenting an input image 
into a plurality of regions, (b) obtaining color vectors for the segmented regions, (c) 
20 classifying the color vectors to obtain dominant colors of the input image and the 
ratios thereof, and (d) representing the dominant colors and the ratios thereof as a 
color feature descriptor of the input image. 

The present invention also provides a computer readable medium having 
program codes executable by a computer to perform a color image processing method 
25 for retrieving a color feature descriptor for describing color features of an image, the 
method comprising the steps of (a) segmenting an input image into a plurality of 
regions, (b) obtaining color vectors for the segmented regions, (c) classifying the 
color vectors to obtain dominant colors of the input image and the ratios thereof, and 
(d) representing the dominant colors and the ratios thereof as a color feature 
30 descriptor of the input image. 

According to another aspect of the present invention, there is provided a color 
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image processing apparatus for retrieving a color feature descriptor for describing 
color features of an image, including a color vector retrieving unit for receiving pixel 
value data of an input image and retrieving color vectors for a predetermined color 
coordinate system, and a color feature descriptor generating unit for obtaining the 
5 percentiles of dominant colors represented by the color vectors when the color 
vectors are all received and generating and outputting color feature descriptor data 
containing the information on the dominant colors and the percentiles thereof. 

Also, the present invention provides a color image processing apparatus for 
retrieving a color feature descriptor for describing color features of an image, 

10 including .a segmenting unit for segmenting an input image into k regions, wherein 
k is an arbitrary positive integer, and sequentially outputting pixel value data 
corresponding to the Jfcth region, a color vector retrieving unit for receiving pixel 
value data of an input image and retrieving color vectors for a predetermined color 
coordinate system, and a color feature descriptor generating unit for obtaining the 

15 percentiles of dominant colors represented by the color vectors when the color 
vectors are all received and generating and outputting color feature descriptor data 
containing the information on the dominant colors and the percentiles thereof. 

20 Brief Description of the Drawings 

The above objects and advantages of the present invention will become more 
apparent by describing in detail preferred embodiments thereof with reference to the 
attached drawings in which: 

FIG. 1 is a flow diagram showing a color image processing method according 
25 to the present invention; 

FIG. 2 is a diagram illustrating image segmentation performed in the step 106 
of FIG. 1; 

FIG. 3 is a block diagram of a color image processing apparatus according 
to the present invention; and 
30 FIGS. 4 A and 4B show the result obtained by performing a region-based 

search with respect to images indexed by a computer program based on the color 
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image processing method according to the present invention. 



5 



Best mode for carrying out the Invention 

Hereinafter, embodiments of the present invention will be described in detail 



with reference to the accompanying drawings. 

Referring to FIG. 1 illustrating a color image processing method according 
to the present invention, a color image A is input (step 100). The color image is 
segmented into a plurality of regions F„ F 2 , F 3 and F 4 (step 102). The segmentation 

10 can be performed based on edge flow, for example. Then, quantized color vectors 
for the respective regions F^ F 2 , F 3 and F 4 are obtained (step 104). 

The step of obtaining the quantized color vectors preferably includes the 
following steps. First, a predetermined filtering step for smoothing and noise 
removal of an image is performed as a pre-processing step. Next, the probability of 

15 pixels in the filtered image being noisy pixels is analyzed to then apply an 
appropriate weight to the same. The probability of pixels in the filtered image being 
noisy pixels is obtained by the color distance from neighboring pixels. For example, 
i pixels, in which / is an arbitrary integer, ranging from a pixel having the minimum 
color distance are selected among the pixels sorted according to the color distance 

20 from a central pixel, and among the selected pixels, the pixel value which has the 
largest color distance is set to the maximum color distance, which is denoted by T(n). 
Then, the color vectors of the respective pixels are weighted by exp(-T(n)). 
exp(-T(n)) is defined by v(n). Next, assuming that the average of T(n) values of all 
pixels is Tavg, the number Nof initial clusters to be used in quantization equals Tavg 

25 x an arbitrary constant, e.g., 2. Then, a general Lloyd algorithm is applied to the 
color vectors corresponding to weighted pixels to quantize the color vectors. First, 
using the cluster centroid (c,) represented by Expression (1): 



wherein X(n) is the pixel value of the nth pixel among the sorted pixels, and a value 




(1) 
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of Di represented by Expression (2) is calculated: 
DrSv^W^f ( 2 ) 

to then split a cluster having the largest value of D,. This procedure is repeated until 
N clusters are generated. After N clusters are generated, a general Lloyd algorithm 

5 is performed. When the general Lloyd algorithm is performed, the cluster centroid 
is calculated by the Expression (1) to perform updating. 

Next, clusters having similar color vectors are agglomerated by performing 
agglomerative clustering. Agglomerative clustering is disclosed by R.O. Duda and 
P.E. Hart in "Pattern Classification and Scene Analysis, John Wiley and Sons, New 

10 York, 1973," which will not be described in detail in this specification. 

Then, the color vectors are classified and dominant colors represented by 
color vectors [cL it cU it cVJ and their percentiles P, are obtained (step 106). Here, 
i denotes the arbitrary serial number of primary regions, ranging from 1 through iV, 
L, U and V denote coordinates of the CIE LUV color coordinate system. The 

15 percentiles P, are expressed by decimals. The sum of the percentiles P, for i regions 
is 1 as represented by Expression (3): 

N 

E'W < 3 >- 

Next, the dominant colors represented by color vectors [cL x , cU it cV it J and 
their percentiles P t are expressed as the color feature descriptor of a pertinent region. 
20 obtained (step 108). In other words, the color feature descriptor F can be 
represented by Expression (4): 

F={{[cL i ,cU i ,cV i ],P i },i=l,...,N} (4) 

wherein N is a predetermined positive integer. The color feature descriptor can be 
referred to as a variable-bin color histogram. 
25 By combining pixel value data in the kth region, i.e., Region k and color 

feature descriptor data of this region, i.e., F k , the whole image A 'is represented by 
Expression (5): 

A'= {Region,,^; Region,, F 2 ;...;Region k ,FJ (5) 
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wherein k is a predetermined positive integer representing the number of segmented 
regions of the image A (step 110). 

The color feature descriptor retrieved by the color image processing method 
according to the present invention is compactly represented by a small number of 

5 numbers with respect to one region. The compact representation of the color feature 
descriptor can remarkably reduce the computational complexity. This allows fast 
search and retrieval of multi-media based contents. The color image processing 
method according to the present invention can be applied to an object-based image 
processing method such as MPEG-7. 

10 The color image processing method is programmable by a computer program. 

Codes and code segments constituting the computer program can be easily derived 
by a computer programmer in the art. Also, the program is stored in computer 
readable media and is readable and executable by the computer, thereby embodying 
the color image processing method. The media include magnetic recording media, 

15 optical recording media, carrier wave media, and the like. 

Also, the color image processing method can be implemented on a color 
image processing apparatus. FIG. 3 is a block diagram of a color image processing 
apparatus according to the present invention. Referring to FIG. 3, the color image 
processing apparatus includes a segmenting unit 300, a color vector retrieving unit 

20 302, a color feature descriptor generating unit 304 and a combining unit 306. 

In the operation of the color image processing apparatus, the segmenting unit 
300 segments an input image A into k regions and sequentially outputs pixel value 
data Region k in the Jkth region. The color vector retrieving unit 302 receives the pixel 
value data Region k in the fcth region and retrieves the color vectors [cL it cU t , cVJ. 

25 When i color vectors [cL if cU it cVJ are all received, the color feature descriptor 
generating unit 304 obtains the percentiles P, of dominant colors represented by the 
color vectors /cL,, cU (t cVJ, and generates and outputs color feature descriptor data 
F k . The color feature descriptor data F k includes information on the dominant colors 
represented by the color vectors fcL if cU lf cVJ and their percentiles P i: 

30 In order to obtain the percentiles P ; of the respective colors, it is more 

preferable that color quantization is performed within each segmented region. Thus, 
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the color image processing apparatus preferably further includes a quantizing unit 
(not shown). The color image processing apparatus preferably further includes a 
filtering unit (not shown) for performing a predetermined filtering process for 
smoothing and noise removal of the input image. The quantizing unit analyzes the 

5 probability of pixels in the filtered image being noisy pixels, applies an appropriate 
weight thereto and quantizes the color vectors corresponding to the weighted pixels 
by a general Lloyd algorithm. 

The combining unit 306 combines pixel value data in the kth region, i.e., 
Region k and color feature descriptor data of this region, i.e., F k , to output the 

10 processed image A ' The color image processing apparatus according to the present 
invention can be applied to an object-based image processing method such as MPEG- 
7. Also, in the color image processing apparatus according to the present invention, 
expressing a color image using dominant colors of the image can also be applied to 
various other fields besides the field of color image processing. 

15 As described above, the color image processing method according to the 

present invention is applied to an object-based image processing method, thereby 
allowing fast search and retrieval of multi-media contents. 

20 Industrial Applicability 

The present invention can be applied to the fields of object-based image 

processing. 
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What is claimed is: 

1. A color image processing method for retrieving a color feature 
descriptor for describing color features of an image, the method comprising the steps 
of: 

5 (a) obtaining color vectors of an input image; 

(b) classifying the color vectors to obtain dominant colors of the input image 
and the ratios thereof; and 

(c) representing the dominant colors and the ratios thereof as a color feature 
descriptor of the input image. 

10 

2 . The color image processing method according to claim 1 , wherein the 
color vectors are quantized color vectors. 

3 . The color image processing method according to claim 1 , wherein the 
15 ratios are percentiles. 

4. The color image processing method according to claim 1, further 
comprising the step of (d) combining the color vectors and the color feature 
descriptor and representing the combination result as the whole image. 

20 

5 . The color image processing method according to claim 4, wherein the 
color vectors are quantized color vectors. 

6. The color image processing method according to claim 4, wherein the 
25 ratios are percentiles. 

7. The color image processing method according to claim 1, further 
comprising the step of (e) combining the quantized color vectors and the color feature 
descriptor and representing the combination result as the whole image. 

30 

8. The color image processing method according to claim 1, before the 
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step (b), further comprising the step of performing a predetermined filtering process 
for smoothing of the input image. 

9. The color image processing method according to claim 1 , before the 
5 step (b), further comprising the step of performing a predetermined filtering process 

for noise removal of the input image. 

10. The color image processing method according to claim 1, before the 
step (b), further comprising the step of performing a predetermined filtering process 

10 for smoothing and noise removal of the input image. 

11. The color image processing method according to any of claims 1 
through 10, before the step (b), further comprising the steps of: 

analyzing the probability of pixels in the filtered image being noisy pixels and 
15 applying appropriate weights thereto; and 

applying a general Lloyd algorithm to the color vectors corresponding to the 
weighted pixels to perform color quantization. 

12. A color image processing method for retrieving a color feature 
20 descriptor for describing color features of an image, the method comprising the steps 

of: 

(a) segmenting an input image into a plurality of regions; 

(b) obtaining color vectors for the segmented regions; 

(c) classifying the color vectors to obtain dominant colors of the input image 
25 and the ratios thereof; and 

(d) representing the dominant colors and the ratios thereof as a color feature 
descriptor of the input image. 

13 . The color image processing method according to claim 12, wherein 
30 the color vectors are quantized color vectors. 
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14. The color image processing method according to claim 12, wherein 
the ratios are percentiles. 

15. The color image processing method according to claim 1, farther 
5 comprising the step of (e) combining the quantized color vectors for the segmented 

regions and the color feature descriptor and representing the combination result as 
the whole image. 

16. The color image processing method according to claim 12, before the 
10 step (b), further comprising the step of performing a predetermined filtering process 

for smoothing of the input image. 

17. The color image processing method according to claim 12, before the 
step (b), further comprising the step of performing a predetermined filtering process 

15 for noise removal of the input image. 

18. The color image processing method according to claim 12, before the 
step (b), further comprising the step of performing a predetermined filtering process 
for smoothing and noise removal of the input image. 

20 

19. The color image processing method according to claim 12, before the 
step (b), further comprising the step of performing a predetermined filtering process 
for smoothing of the segmented regions. 

25 20. The color image processing method according to claim 12, before the 

step (b), further comprising the step of performing a predetermined filtering process 
for noise removal of the segmented regions. 

21 . The color image processing method according to claim 12, before the 
30 step (b), further comprising the step of performing a predetermined filtering process 
for smoothing and noise removal of the segmented regions. 
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22. The color image processing method according to any of claims 12 
through 21, before the step (b), further comprising the steps of: 

analyzing the probability of pixels in the filtered image being noisy pixels and 
applying appropriate weights thereto; and 
5 applying a general Lloyd algorithm to the color vectors corresponding to the 

weighted pixels to perform color quantization. 

23. A computer readable medium having program codes executable by a 
computer to perform a color image processing method for retrieving a color feature 

10 descriptor for describing color features of an image, the method comprising the steps 

of: 

(a) segmenting an input image into a plurality of regions; 

(b) obtaining color vectors for the segmented regions; 

(c) classifying the color vectors to obtain dominant colors of the input image 
15 and the ratios thereof; and 

(d) representing the dominant colors and the ratios thereof as a color feature 
descriptor of the input image. 

24. The computer readable medium according to claim 23, further 
20 comprising the step of (e) combining the quantized color vectors for the segmented 

regions and the color feature descriptor and representing the combination result as 
the whole image. 

25. A color image processing apparatus for retrieving a color feature 
25 descriptor for describing color features of an image, comprising: 

a color vector retrieving unit for receiving pixel value data of an input image 
and retrieving color vectors for a predetermined color coordinate system; and 

a color feature descriptor generating unit for obtaining the percentiles of 
dominant colors represented by the color vectors when the color vectors are all 
30 received and generating and outputting color feature descriptor data containing the 
information on the dominant colors and the percentiles thereof. 
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26. The color image processing apparatus according to claim 25, further 
comprising combining unit for combining pixel value data and color feature 
descriptor data to output a processed image. 

5 27. A color image processing apparatus for retrieving a color feature 

descriptor for describing color features of an image, comprising: 

a segmenting unit for segmenting an input image into k regions, wherein k is 
an arbitrary positive integer, and sequentially outputting pixel value data 
corresponding to the kth region; 
10 a color vector retrieving unit for receiving pixel value data of an input image 

and retrieving color vectors for a predetermined color coordinate system; and 

a color feature descriptor generating unit for obtaining the percentiles of 
dominant colors represented by the color vectors when the color vectors are all 
received and generating and outputting color feature descriptor data containing the 
15 information on the dominant colors and the percentiles thereof. 

28. The color image processing apparatus according to claim 27, further 
comprising a quantizing unit for performing color quantization in the segmented 
region. 

20 

29. The color image processing apparatus according to claim 28, wherein 
the quantizing unit analyzes the probability of pixels in the filtered image being noisy 
pixels and applying appropriate weights thereto, and applies a general Lloyd 
algorithm to the color vectors corresponding to the weighted pixels to perform color 

25 quantization. 

30. The color image processing apparatus according to claim 27, further 
comprising a combining unit for combining pixel value data corresponding to the kth 
region and color feature descriptor data of the corresponding region, with respect to 

30 all the k segmented regions, to output a processed image. 
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31 . The color image processing apparatus according to claim 30, further 
comprising a quantizing unit for performing color quantization in the segmented 
regions. 

5 32 . The color image processing apparatus according to claim 3 1 , wherein 

the quantizing unit analyzes the probability of pixels in the filtered image being noisy 
pixels and applying appropriate weights thereto, and applies a general Lloyd 
algorithm to the color vectors corresponding to the weighted pixels to perform color 
quantization. 

10 

33. The color image processing apparatus according to any of claims 27 
through 32, further comprising a filtering unit for performing a predetermined 
filtering process for smoothing of an input image. 

15 34. The color image processing apparatus according to any of claims 27 

through 32, further comprising a filtering unit for performing a predetermined 
filtering process for noise removal of an input image. 

35. The color image processing apparatus according to any of claims 27 
20 through 32, further comprising a filtering unit for performing a predetermined 

filtering process for smoothing and noise removal of an input image. 

36. The color image processing apparatus according to any of claims 33 
through 35, further comprising a quantizing unit for performing color quantization 

25 in the segmented regions. 

37. The color image processing apparatus according to claim 36, wherein 
the quantizing unit analyzes the probability of pixels in the filtered image being noisy 
pixels and applying appropriate weights thereto, and applies a general Lloyd 

30 algorithm to the color vectors corresponding to the weighted pixels to perform color 
quantization. 
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38. The color image processing apparatus according to claim 37, further 
comprising a combining unit for combining pixel value data corresponding to the *th 
35 region and color feature descriptor data of the corresponding region, with respect to 
all the k segmented regions, to output a processed image. 



39. A method for representing a color image, wherein the color image is 
represented using dominant colors of the color image and the percentiles thereof. 



WO 00/46748 



PCT/KROO/00089 



1/4 

FIG. 1 

cp 

—100 
[ —102 

—104 



— 106 



—108 

I 



CD 



WO 00/46748 



PCT/KROO/00089 



2/4 

FIG. 2 



Fi 




WO 00/46748 



PCT/KR00/00089 



3/4 



CO 

6 



t- CM 

II Ll Ll 

^ CN 

c c c 

o o o 

*cr> "cr> cj» 

<U (U (D 

d o: d 




o 
I — o 



WO 00/46748 



PCT/KR00/00089 




INTERNATIONAL SEARCH REPORT 



International application No. 
PCT/KR00/00089 



A. CLASSIFICATION OF SUBJECT MATTER , 

IPC7 G06T 1/00, G06T 7/60 
According to International Patent Classification (IPC) or to both national classification and IPC 

B. FIELDS SEARCHED 

Minimun documentation searched (classification system followed by classification symbols) 
rPC7 G06T1/00, G06T 7/00, G06F 17/30 

Documentation searched other than minimun documentation to the extent that such documents arc included in the filcds searched 
Korean patents and applications for inventions since 1975 
Korean Utility models and applications for Utility models since 1975 

Electronic data base consulted during the intertnational search (name of data base and, where practicable, search trcrms used) 
WPI, PAJ, TFraiFK Electronic Library (Since 1988) "COLOR", "VECTOR", "RETRIEVE". "IMAGE" 



C. DOCUMENTS CONSIDERED TO BE RELEVANT 



Category* 


Citation of document, with indication, where appropriate, of the relevant passages 


Relevant to claim No. 


Y 


JP 1 1-238077 A (MINOLTA CO., LTD) 31 August 1999 (31. 03. 1999) 
The whole document 


1-6, 12-14,23-28 


Y 


KR 98-022999 A (DAEWOO ELECTRONICS CO., LTD) 6 July 1998 (06. 07. 1998) 
The whole document 


1-6, 12-14,23-28 


Y 


Acoustics, Speech, and Signal Processing. 1996. ICASSP-96. Conference Proceedings., 1996 
IEEE International Conference on Published: 1996 Page(s): 167 -173 


1-6, 12-14,23-28 




Ardtzzoni, S.; Bartolini, L; Patella, M. 

"Windsurf: region-based image retrieval using wavelets" 

The whole document 




A 


Acoustics, Speech, and Signal Processing. 1999. Proceedings., 1999 IEEE International 
Conference on Published: 1999 Volume: 6 , Page(s): 3017 -3020 vol.6 ' 
Yining Deng; Manjunath, B.S. 

"An efficient low-dimensional color indexing scheme for region-based image retrieval " 
The whole document 


MO, 12-20,23-28, 
30-31,33-36, 37-39 



| I Further documents are listed in the continuation of Box C. Q See patent family annex. 



♦ Special categories of cited documents: 

"A" document defining the general sUte of (he art which is not considered 

to be of particular relevence 
"E" earlier application or potent but published on or after the international 

filing date 

"L" document which may throw doubts on priority c!atm(s) or which is 
cited to establish the publication date of citation or other 
special reason (as specified) 

"O m document referring to an oral disclosure, use. exhibition or other 
means 

"P" document published prior to the international Tiling date but later 



T later document published after the international filing date or priority 

date and not in conflict with the application but cited to understand 

the principle or theory underlying the invention 
"X" document of particular relevence; the claimed invention cannot be 

considered novel or cannot be considered to involve an inventive 

step when the document is taken alone 
"Y" document of particular relevence; the claimed invention cannot be 

considered to involve an inventive step when the document is 

combined with one or more other such documents^ ch combination 

being obvious to a person skilled in the art 

document member of the same patent family 



Date of the actual completion of the international search 
IS MAY 2000 (15.05.2000) 


Date of mailing of the international search report 
18 MAY 2000(18.05.2000) 


Name and mailing address of the ISA/KR 
Korean Industrial Property Office 

Government Complcx-Taejon. Dunsan-dong, So-ku. Taejon 
Metropolitan City 302-701. Republic of Korea 

Facsimile No. 82-42-472-7140 


Authorized officer *v 

LEE, Jung Suk *j J 
Telephone No. 82-42-481-5789 VtZL-'* 



Form PCT/KA/2 10 (second sheet) (July 1998) 



INTERNATIONAL SEARCH REPORT 


International application No. 
PCT/KROO/00089 


C (Continuation). DOCUMENTS CONSIDERED TO BE RELEVANT 




Category* 


Citation of document, with indication, where appropriate, of the relevant passages 


Relevant to claim No. 


Y 


JP 06-318256 A (MATSUSHITA ELECTRIC IND CO., LTD) 15 November 1994 

(15.11.1994) 

The whole document 


1-6, 12-14,23-28 


A 


JP 05-266091 A (MITSUBISHI ELECTRIC CO.) 15 October 1993 (15. 10. 1993) 
The whole document 


MO, 12-20,23-28, 
30-31,33-36, 37-39 



Form PCT/IS/W210 (continuation of second sheet) (July 1998) 



